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1 Introduction. 



The phrase "Information is physical", enunciated by Rolf Landauer jT] is 
the core of the research field known as physics of information. The author 
claimed that information is everywhere, and, whether it is in biological sys- 
tems, in a computer, or in a black hole, inevitably has a physical form. Also 
information, of whatever kind, is associated with matter, radiation or fields 
of different type. It is impossible to get away from this sort of physical em- 
bodiment. Therefore, manipulation of information is inevitably subject to 
the laws of physics. 

On the other hand there is a particular physical conjecture known as 
the Holographic Principle[2], which states that the number of the degrees of 
freedom describing the physics inside a volume (including gravity) is bounded 
by the area of the boundary which encloses this volume. 

As information is physical we can also try the other way around, inter- 
preting physical theories in terms of the language of the Information Theory. 
More precisely, in this paper I will attempt to connect the holography with 
one of the cornerstone of Information Theory, the Shannon noiseless coding 
theorem|2]. 

2 Information Theory . 

The classical theory of information is due to a Bell Labs mathematician, 
Claude Shannon, who in two seminal works definitively laid down its prin- 
ciples. The works of Shannon deal with two very important tasks of infor- 
mation processing :(i)file compression and (ii)error correction. The Shannon 
noiseless coding theorem -also called source coding theorem- addresses the 
first issue: How much redundancy can be eliminated from a given source text 
without losing information? The Shannon noisy coding theorem addresses 
the second issue: How much redundancy must be added to a given source 
text in order to guarantee error free communication over a noisy channel. 
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In this paper we will concentrate just on the first theorem and its relation 
with holography. 

2.1 Definitions and example 

Information is discretized: it comes in irreducible packages. The elementary 
unit of classical information is the bit(or cbit, for classic bit), a classical 
system with only two states, and 1. Any text can be coded into a string of 
bitsH, ISJ. 

Definitions 

• Let S := {si, S2, s„} be a finite alphabet, endowed with a probability 
distribution P : Sj ^ Ps{sj) 

• Codeword - a string of bits which represents a symbol in the alphabet. 

• Code, codebook - the set of all possible codewords, e.g., the 4-bit binary 
code has 16 code-words 

• Let F = {S, P) be a source. An encoding scheme for F is an ordered 
pair (C, /), where C is a code and / : S" — C is an injective function, 
called encoding function. 

For the purposes of noiseless encoding, the measure of efficiency of an enco- 
ding scheme is its average codeword length. 

Definition: The average codeword length of an encoding scheme (C, /) 
for a source F = (S*, P) is defined by 

N 

AveLen{CJ) = (1) 

i=l 

where len{f{si)) is the length of the string that represents Sj. 

The Noiseless Coding Theorem, first proved by Claude Shannon in 1948, 
asserts that the average codeword length of any instantaneous code is bounded 
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from below by H{C), \.e.H{C) < AveLen{C), where H is the so called 
Shannon's entropy 

H{C) = -j2P^log2P^ (2) 
1=1 

The randomness of an information source can be describe by its entropy. The 
operational meaning of entropy is that it determines the smallest number of 
bits per symbol that is required to represent the total output. 
Example 

Consider an alphabet {a, 6, c, d} with probabilities {|, |, |, |}. With four 
alternatives, it seems natural to assign two-bit binary codewords to each sym- 
bol, eg, {00, 01, 10, 11}. If this coding scheme is used, the average codeword 
length is 

1111 
AveLenil) = -2 + -2 + -2 + -2 = 2. (3) 
2 4 8 8 

However, we can do better than this. The entropy of this alphabet is 

H^- [^-log,- + -log,- + -log,- + -log,-) = - (4) 

and Shannon's theorem says that a code exists which achieves this rate. In 
this case, it is easy. The code {0, 10, 110, 111} assigns shorter codewords to 
the more frequent symbols. The average codeword length is 

AveLenil) = il + I2 + ^3 + ^3 = ^. (5) 

In the simple example above, the Shannon entropy and the average code 
length were both 7/4 bits/symbol. It happened this way because the pro- 
babihties were powers of 1/2. We are not so fortunate in reahty and there 
will always be a difference between the Shannon's entropy and the achieved 
average code length. This is quantified by two numbers, the efficiency and 
the redundancy 

Efficiency = - — ; — < 1 (6) 

AveLen 

Redundancy = 1 — Efficiency. (7) 
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3 Holography 



Physicists have the standard vision that the degrees of freedom of the world 
consist of fields filling space. Improving this scenario some theorists pro- 
posed that a small distance cutoff will be required in order to make sense 
of quantum gravity. Therefore according to that hypothesis the world could 
be seen as three dimensional discrete lattice with spacing order of the Plank 
length, Ip. But recently, 't Hooft and Susskind(2I suggested an idea even 
more audacious. According to them the combination of Quantum Mechanics 
and Gravity requires the three dimensional world to be an image of data that 
can be stored on a two dimensional projection much like a holographic im- 
age. This description only requires one discrete degree of freedom per Planck 
area and yet it is rich enough to describe all three dimensional phenomena. 
Therefore it is obvious that the assumption of these ideas imply a radical 
decrease in the number of degrees of freedom for describing the Universe. 

The inspiration for the holographic principle comes from the black hole 
physics where the entropy of a black hole is given by the Bekenstein-Hawking 
formula 

S-^-^'o,2, (8) 

where A is the horizon area. This result was obtained when they realized the 
striking resemblance between the laws of black hole mechanics and the laws 
of thermodynamics. Later, a great amount of work has been done in order 
to find a precise statistical mechanical interpretation of black hole entropy. 
One could derive the Bekenstein-Hawking formula by counting black hole 
microstates and using the Boltzmann entropy formula 

S = lnW, (9) 

where W is the number of microstates of the system. 

This counting is not a simple task because is quite difficult to identified 
the black hole microstates. However great progress in this direction has been 
reported [H]. 
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Therefore, the basic idea of holography is that the entropy scales like 
area, or in other words, the total number of states is of order 

Wa ~ e^. (10) 

This is a very dramatic change of view because, from the quantum field 
theory experience, generally one expects that if the energy density is bounded 
then the maximum entropy is proportional to the volume of space and this 
means 

Wv ~ e^. (11) 

Therefore, for our purposes, the most important conclusion that we can 
extract from the holographic principle is that the radical reduction of the de- 
grees of freedom necessary for the description of the universe can be expressed 
in the following simple way, 

Wv ^ Wa. (12) 



4 Holography as the best Code 

The tools and concepts developed in the above sections will help to show the 
main idea of this work. Thus, the language of Information Theory will be 
used to deal with Holography. 

Firstly we should point out the relation between the Boltzmann and Shan- 
non formulas. Despite the fact that they had different origins they are con- 
ceptually equivalent jH] and the formal relation is very simple,^ 

S = H. (13) 

log2e 

Here we ignore the units of the thermodynamical entropy and we consider 
just the bit units. 



^The relation can be obtained when in (O the outcomes are equally likely, that is, when 



Pi — w- 
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This relation allows to establish the proper bridge between the two main 
topics of this work. 

Now we arrive to our point. First we consider a gravitational system with 
a corresponding 'physical alphabet' F. This 'alphabet' can be thought as the 
source of the dynamic of the system. The next step is to consider the codes. 

Let us suppose that there exits an 'holographic code' A for this gravita- 
tional system and the corresponding average codeword length is 

AveLen{A) = —log2e. (14) 

Similarly, if instead of that we take a 'volumetric code'V, we assume that 

AveLeniY) = V log2e. (15) 

On the other hand, if the Holographic Principle is satisfied, the Boltzmann 
entropy is S' = A/ 4. Therefore, using (fTSj) we get 

H = jlog2e. (16) 

Obtaining, 

H = AveLen{A). (17) 

Consequently, with the support of the Noiseless Coding Theorem we can 
conclude that the 'holografic code' is the best encoding scheme that can be 
constructed. 

In this context a volumetric quantum field theory can be seen as a highly 
inefficient encoding scheme where 

Efficiency = - ~ 1/i? < 1. (18) 

5 Conclusions 

Fermat pointed out that nature is economical, and that light will therefore 
travel along the path that takes the least time. This is quite a good rule but 
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there are some instances where it is erroneous and the modern statement is 
extended: the ray path along which hght travels is such that the time taken 
holds a stationary value (i.e. minimum, maximum or constant). Extending 
this argument to physics of information we could speculate that Holography 
is a manifestation of the economical character of nature. The minimum 
amount of information is used to describe the Universe. 

However, that is not the end of the story. We can go on further in the same 
direction studying also the possible connections of Holography with the Shan- 
non's Second Theorem. Also the arguments presented here could be refined 
using the Quantum Source Coding Theorem, introduced by Schumacher |lUj 

Taking into account that the main idea of this work is encoded on the title 
I will finish with the hope that I introduced the less possible redundancy. 
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